{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/jermwatt/machine_learning_refined/blob/main/notes/9_Feature_engineer_select/9_7_Regularization.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6MAXloi10y-I"
   },
   "source": [
    "## Chapter 9: Principles of Feature Engineering and Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook contains interactive content from an early draft of the university textbook <a href=\"https://github.com/neonwatty/machine-learning-refined/tree/main\">\n",
    "Machine Learning Refined (2nd edition) </a>.\n",
    "\n",
    "The final draft significantly expands on this content and is available for <a href=\"https://github.com/neonwatty/machine-learning-refined/tree/main/chapter_pdfs\"> download as a PDF here</a>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Cn-pkeoR0y-K"
   },
   "source": [
    "# 9.7 Feature Selection via Regularization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ePSp2cVg0y-K"
   },
   "source": [
    "With the first approach to feature selection discussed in the previous Section - *boosting* - we took a 'bottom-up' approach to feature selection: that is we began by tuning the bias and then added new features to our model one-at-a-time.  In this Section we introduce a complementary approach - called *regularization*.  Instead of building up a model 'starting at the bottom', with regularization we take a 'top-down' view and start off with a complete model that includes every one of our input features, and then we gradually remove input features of inferior importance.  We do this by adding a second function to our cost - called a *regularizer*  - that penalizes weights associated with less important input features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "dOOAu6HY0y-L"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: github-clone in /Users/jeremywatt/Desktop/machine-learning-refined/venv/lib/python3.10/site-packages (1.2.0)\n",
      "Requirement already satisfied: requests>=2.20.0 in /Users/jeremywatt/Desktop/machine-learning-refined/venv/lib/python3.10/site-packages (from github-clone) (2.32.3)\n",
      "Requirement already satisfied: docopt>=0.6.2 in /Users/jeremywatt/Desktop/machine-learning-refined/venv/lib/python3.10/site-packages (from github-clone) (0.6.2)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /Users/jeremywatt/Desktop/machine-learning-refined/venv/lib/python3.10/site-packages (from requests>=2.20.0->github-clone) (3.4.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /Users/jeremywatt/Desktop/machine-learning-refined/venv/lib/python3.10/site-packages (from requests>=2.20.0->github-clone) (3.10)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/jeremywatt/Desktop/machine-learning-refined/venv/lib/python3.10/site-packages (from requests>=2.20.0->github-clone) (2.2.3)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /Users/jeremywatt/Desktop/machine-learning-refined/venv/lib/python3.10/site-packages (from requests>=2.20.0->github-clone) (2024.12.14)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "chapter_9_datasets already cloned!\n",
      "chapter_9_library already cloned!\n",
      "chapter_9_images already cloned!\n",
      "chapter_9_videos already cloned!\n"
     ]
    }
   ],
   "source": [
    "# install github clone - allows for easy cloning of subdirectories\n",
    "!pip install github-clone\n",
    "from pathlib import Path \n",
    "\n",
    "# clone datasets\n",
    "if not Path('chapter_9_datasets').is_dir():\n",
    "    !ghclone https://github.com/neonwatty/machine-learning-refined-notes-assets/tree/main/notes/9_Feature_engineer_select/chapter_9_datasets\n",
    "else:\n",
    "    print('chapter_9_datasets already cloned!')\n",
    "\n",
    "# clone library subdirectory\n",
    "if not Path('chapter_9_library').is_dir():\n",
    "    !ghclone https://github.com/neonwatty/machine-learning-refined-notes-assets/tree/main/notes/9_Feature_engineer_select/chapter_9_library\n",
    "else:\n",
    "    print('chapter_9_library already cloned!')\n",
    "\n",
    "# clone images\n",
    "if not Path('chapter_9_images').is_dir():\n",
    "    !ghclone https://github.com/neonwatty/machine-learning-refined-notes-assets/tree/main/notes/9_Feature_engineer_select/chapter_9_images\n",
    "else:\n",
    "    print('chapter_9_images already cloned!')\n",
    "\n",
    "# clone videos\n",
    "if not Path('chapter_9_videos').is_dir():\n",
    "    !ghclone https://github.com/neonwatty/machine-learning-refined-notes-assets/tree/main/notes/9_Feature_engineer_select/chapter_9_videos\n",
    "else:\n",
    "    print('chapter_9_videos already cloned!')\n",
    "\n",
    "# append path for local library, data, and image import\n",
    "import sys\n",
    "sys.path.append('./chapter_9_library') \n",
    "\n",
    "# import section helper\n",
    "import section_9_7_helpers\n",
    "\n",
    "# dataset paths \n",
    "data_path_1 = 'chapter_9_datasets/credit_dataset.csv'\n",
    "\n",
    "# video paths\n",
    "video_path_1 = 'chapter_9_videos/animation_2.mp4'\n",
    "\n",
    "# standard imports\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.display import Image, HTML\n",
    "import autograd.numpy as np\n",
    "from base64 import b64encode\n",
    "\n",
    "def show_video(video_path, width = 1000):\n",
    "    video_file = open(video_path, \"r+b\").read()\n",
    "    video_url = f\"data:video/mp4;base64,{b64encode(video_file).decode()}\"\n",
    "    return HTML(f\"\"\"<video width={width} controls><source src=\"{video_url}\"></video>\"\"\")\n",
    "\n",
    "# this is needed to compensate for matplotlib notebook's tendancy to blow up images when plotted inline\n",
    "%matplotlib inline\n",
    "from matplotlib import rcParams\n",
    "rcParams['figure.autolayout'] = True\n",
    "\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GA_yaiMe0y-N"
   },
   "source": [
    "## Regularization using simple vector norms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6tnqywgV0y-N"
   },
   "source": [
    "The simple linear combination of two functions $f_1$ and $f_2$ \n",
    "\n",
    "\\begin{equation}\n",
    "f\\left(\\mathbf{w}\\right) = f_1(\\mathbf{w})+\\lambda\\,\\,f_2(\\mathbf{w})\n",
    "\\end{equation}\n",
    "\n",
    "is often referred to as *regularization* in the parlance of machine learning.  More specifically the function $f_2$ is called a *regularizer* in the jargon of machine learning, since by adding it to function $f_1$ we adjust its shape, and $\\lambda \\geq 0$ is referred to as a penalty or regularization parameter (we have already seen one instance of regularization in discussing how to adjust Newton's method to deal with non-convex functions in Chapter 5).  \n",
    "\n",
    "Here when $\\lambda = 0$ the above combination reduces to $f_1$, and as we increase $\\lambda$ the two functions $f_1$ and $f_2$ 'compete for dominance' in a linear combination of the two functions which takes on properties of both functions (see the book Appendix for several visual examples).  As we set $\\lambda$ to larger and larger values the function $f_2$ dominates the combination, and eventually completely drowns out function $f_1$.  In this instance what we end up with is a highly positively scaled version of the regularizer $f_2$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NE63POxb0y-N"
   },
   "source": [
    "Suppose we added a generic function $h\\left(\\cdot\\right)$ to one of our supervised learning functions e.g., the Least Squares $g\\left(\\mathbf{w}\\right) = \\frac{1}{P}\\sum_{p=1}^{P}\\left(\\text{model}\\left(\\mathbf{x}_p^{\\,},\\mathbf{w}\\right)  - \\overset{\\,}{y}_{p}^{\\,}\\right)^{2}$, giving a regularized linear combination of our cost function\n",
    "\n",
    "\\begin{equation}\n",
    "f\\left(\\mathbf{w}\\right) = g\\left(\\mathbf{w}\\right) + \\lambda \\, h\\left(\\mathbf{w}\\right).\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DrYBJHsg0y-O"
   },
   "source": [
    "By adding the regularizer $h\\left(\\mathbf{w}\\right)$ to $g$ we alter its very nature, and in particular change the location of its minima (see example 3 in the book Appendix).  The larger we set the regularization parameter $\\lambda$ the more the combination $f$ drifts towards a positively scaled version of the regularizer $h$ with minima that mirror closely those of $h$.\n",
    "\n",
    "By carefully engineering $h$ and the value of $\\lambda$, we can construct a *regularized* form of the original cost function that retains the essence of $g$ - in that it measures a regression error of a model against a set of input/output data points - but whose altered minima better reflect *only the most input features of a dataset*.  One can do this just as easily with a *convex combination* of a cost function and another function $h$ - but the regularization paradigm is more popular in machine learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Kz1ZMLmh0y-O"
   },
   "source": [
    "A common choice for $h$ that achieve these desired aims is a  *vector norm* - in particular the so-called $\\ell_1$ norm.  Remember that a vector norm is a simple function for measuring the *magnitude* or *length* of a vector (see Appendix for further details).  Below we discuss several basic choices of vector norms, the latter two of which are very commonly used in machine learning applications."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0uWqiUK_0y-O"
   },
   "source": [
    "#### <span style=\"color:#a50e3e;\">Example 4: </span>  Regularization using the $\\ell_0$ norm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HbgWsLP10y-O"
   },
   "source": [
    "The $\\ell_0$ vector norm, written as $\\left \\Vert \\mathbf{w} \\right \\Vert_0$, measures 'magnitude' as\n",
    "\n",
    "\\begin{equation}\n",
    "\\left \\Vert \\mathbf{w} \\right \\Vert_0 = \\text{number of non-zero entries of  } \\mathbf{w}.\n",
    "\\end{equation}\n",
    "\n",
    "If we regularize a cost $g$ using it \n",
    "\n",
    "\\begin{equation}\n",
    "f\\left(\\mathbf{w}\\right) = g\\left(\\mathbf{w}\\right) + \\lambda \\,  \\left \\Vert \\mathbf{w} \\right \\Vert_0\n",
    "\\end{equation}\n",
    "\n",
    "notice what we are doing: we are *penalizing* the regularized cost *for every non-zero entry of $\\mathbf{w}$* since every non-zero entry *adds $+1$* to the magnitude $\\left \\Vert \\mathbf{w} \\right \\Vert_0$.  Conversely then, in  *minimizing this sum*, the two functions $g$ and $\\left \\Vert \\mathbf{w} \\right \\Vert_0$ 'compete for dominance' with $g$ wanting $\\mathbf{w}$ to be resolved so that we attain a strong regression fit, while the regularizer $\\left \\Vert \\mathbf{w} \\right \\Vert_0$ *aims to determine a 'small' $\\mathbf{w}$ that has as few non-zero elements as possible* (or - in other words - a weight vector $\\mathbf{w}$ that is very *sparse*). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t3R-L6NS0y-P"
   },
   "source": [
    "#### <span style=\"color:#a50e3e;\">Example 5: </span>  Regularization using the $\\ell_1$ norm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZOBl_LBi0y-P"
   },
   "source": [
    "The $\\ell_1$ vector norm, written as $\\left \\Vert \\mathbf{w} \\right \\Vert_1$, measures 'magnitude' as\n",
    "\n",
    "\\begin{equation}\n",
    "\\left \\Vert \\mathbf{w} \\right \\Vert_1 = \\sum_{n=0}^N \\left \\vert w_n \\right \\vert.\n",
    "\\end{equation}\n",
    "\n",
    "If we regularize a regression cost $g$ (or - as we will see - any machine learning cost) using it \n",
    "\n",
    "\\begin{equation}\n",
    "f\\left(\\mathbf{w}\\right) = g\\left(\\mathbf{w}\\right) + \\lambda \\,  \\left \\Vert \\mathbf{w} \\right \\Vert_1\n",
    "\\end{equation}\n",
    "\n",
    "notice what we are doing: we are *penalizing* the regularized cost *based on the sum of the absolute value of the entries of $\\mathbf{w}$*.  \n",
    "\n",
    "Conversely then, in  *minimizing this sum*, the two functions $g$ and $\\left \\Vert \\mathbf{w} \\right \\Vert_1$ 'compete for dominance' with $g$ wanting $\\mathbf{w}$ to be resolved so that we attain a strong regression fit, while the regularizer $\\left \\Vert \\mathbf{w} \\right \\Vert_1$ aims to determine a $\\mathbf{w}$ that is small both in terms of the absolute value of each of its components but also - because the $\\ell_1$ norm is so closely related to the $\\ell_0$ norm (see Appendix) - one that has few non-zero entries and is quite *sparse*. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2HvDBRpc0y-P"
   },
   "source": [
    "#### <span style=\"color:#a50e3e;\">Example 6: </span>  Regularization using the $\\ell_2$ norm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NALn_c3t0y-Q"
   },
   "source": [
    "The $\\ell_1$ vector norm, written as $\\left \\Vert \\mathbf{w} \\right \\Vert_2$, measures 'magnitude' as\n",
    "\n",
    "\\begin{equation}\n",
    "\\left \\Vert \\mathbf{w} \\right \\Vert_2 = \\sqrt{\\left(\\sum_{n=0}^N w_n^2 \\right )}.\n",
    "\\end{equation}\n",
    "\n",
    "If we regularize a regression cost $g$ (or - as we will see - any machine learning cost) using it\n",
    "\n",
    "\\begin{equation}\n",
    "f\\left(\\mathbf{w}\\right) = g\\left(\\mathbf{w}\\right) + \\lambda \\,  \\left \\Vert \\mathbf{w} \\right \\Vert_2\n",
    "\\end{equation}\n",
    "\n",
    "notice what we are doing: we are *penalizing* the regularized cost *based on the sum of squares of the entries of $\\mathbf{w}$*.  \n",
    "\n",
    "Conversely then, in  *minimizing this sum*, the two functions $g$ and $\\left \\Vert \\mathbf{w} \\right \\Vert_2$ 'compete for dominance' with $g$ wanting $\\mathbf{w}$ to be resolved so that we attain a strong regression fit, while the regularizer $\\left \\Vert \\mathbf{w} \\right \\Vert_2$ aims to determine a $\\mathbf{w}$ that is small in the sense that *all of its entries* have a small *squared* value. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XimX5qbJ0y-Q"
   },
   "source": [
    "##  Feature selection via $\\ell_1$ regularization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "P-NHCQBa0y-Q"
   },
   "source": [
    "From the perspective of machine learning, by inducing the discovery of *sparse minima* the $\\ell_0$ and $\\ell_1$ reguarlizers help uncover the identity of a dataset's *most important features*: these are precisely the features associated with the non-zero feature-touching weights.  This makes both norms - at least in principle - quite appropriate for our current application.  On the other hand, the drive for general 'smallness' induced by minimizing a cost with the $\\ell_2$ regularizer means - provided $\\lambda$ is set correctly - that typically *every* weight is included in the minima recovered.  This makes the $\\ell_2$ intuitively less appealing for the application of feature selection (although it is used throughout machine learning as a regularizer for applications other than feature selection, as we will see)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FyU8boE30y-Q"
   },
   "source": [
    "Of the two sparsity-inducing norms described here the $\\ell_0$ norm - while promoting sparsity to the greatest degree - is also the most challenging to employ since it is *discontinuous* making the minimization of an $\\ell_0$ regularized cost function quite difficult.  While the $\\ell_1$ norm induces sparsity to less of a degree, it is *convex and continuous* and so we have no practical problem minimizing a cost function where it is used as regularizer.  Because of this practical advantage the $\\ell_1$ norm is by far the more commonly used regularizer for feature selection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "id": "O0glHlpe0y-Q"
   },
   "source": [
    "Finally, remember that when performing feature selection we are only interested in determining the importance of *feature-touching* weights $w_1,\\,w_2,\\,...,w_N$ we only need regularize them (and not the bias weight $w_0$), and so our regularization will more specifically take the form\n",
    "\n",
    "\\begin{equation}\n",
    "f\\left(\\mathbf{w}\\right) = g\\left(\\mathbf{w}\\right) + \\lambda \\,  \\sum_{n=1}^N \\left\\vert w_n \\right\\vert.\n",
    "\\end{equation}\n",
    "\n",
    "where $g$ is any supervised learning cost function.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tCUEDbRz0y-R"
   },
   "source": [
    "Using our individual notation for the bias and feature-touching weights \n",
    "\n",
    "\\begin{equation}\n",
    "\\text{(bias):}\\,\\, b = w_0 \\,\\,\\,\\,\\,\\,\\,\\, \\text{(feature-touching weights):} \\,\\,\\,\\,\\,\\, \\boldsymbol{\\omega} = \n",
    "\\begin{bmatrix}\n",
    "w_1 \\\\\n",
    "w_2 \\\\ \n",
    "\\vdots \\\\\n",
    "w_N\n",
    "\\end{bmatrix}.\n",
    "\\end{equation}\n",
    "\n",
    "we can write this general $\\ell_1$ regularized cost function equivalently as\n",
    "\n",
    "\\begin{equation}\n",
    "f\\left(b,\\boldsymbol{\\omega}\\right) = g\\left(b,\\boldsymbol{\\omega}\\right)  + \\lambda \\,  \\left\\Vert  \\boldsymbol{\\omega} \\right\\Vert_1.\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5_QPPaHu0y-R"
   },
   "source": [
    "We have seen feature-touching regularized supervised learning cost functions before, e.g., throughout Chapters 6 and 7."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "N_MQEC710y-R"
   },
   "source": [
    "##  What value should $\\lambda$ be set too?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AnNp_zpw0y-R"
   },
   "source": [
    "Because feature selection is done for the purposes of *human interpretation* the value of $\\lambda$ can be based on several factors.  A benchmark value for $\\lambda$ can be hand chosen based on the desire to explore a dataset, finding a value that provides sufficient sparsity while retaining a low cost value.  Finally, $\\lambda$ can be chosen entirely based on the sample statistics of the dataset via a procedure known as *cross-validation* (which we discuss in Section 12.6)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H3d_YILW0y-R"
   },
   "source": [
    "Regardless of how we select $\\lambda$, note that because we are trying to determine the *importance* of each input feature here that before we begin the process of regularization we *always need to standard normalize the input to our dataset* as detailed in [Section 9.3](https://jermwatt.github.io/machine_learning_refined/notes/9_Feature_engineer_select/9_3_Scaling.html).  By normalizing each input feature distribution we can fairly compare each input feature's contribution and determine the importance of each."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Aeowntox0y-R"
   },
   "source": [
    "#### <span style=\"color:#a50e3e;\">Example 2: </span>  Exploring features for classifying credit risk of loan applicants via $\\ell_1$ regularization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_qAlvW9z0y-S"
   },
   "source": [
    "In this example we use the Credit Risk dataset first introduced in Example 2 of the previous Section.  This dataset consists of $P=1000$ datapoints with $N=20$ input features relating various statistics about loan applicants to their assessed credit risk.  We use $\\ell_1$ regularization with $50$ evenly spaced values for $\\lambda$ in the range $\\left[0, 130 \\right]$.  Shown in the animated figure below is a histogram of the optimal *feature-touching* weights recovered by minimizing an $\\ell_1$ regularized Least Squares cost function with these various $\\lambda$ values.  In each case we run $1000$ steps of gradient descent were taken with $\\alpha = 10^{-1}$ for all runs. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "npqLs1zk0y-S"
   },
   "source": [
    "Moving the slider from its starting position on the left (where $\\lambda = 0$) rightwards increases the value of $\\lambda$ and the associated optimal set of weights are shown.  Pushing the slider all the way to the right shows the optimal weights resulting in using the maximum value for $\\lambda$ used in these experiments.  As you move the slider from left to right you can see how the resulting optimal featre-touching weights begin to *sparsify*, with many of them diminishing to near zero values.  By the time $\\lambda \\approx 40$ five major weights remain, corresponding to feature $1 - 3$ and $6 - 7$.  The first there features were also determined to be important via boosting - and can be easily interpreted. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "Bz73PY590y-S",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "## uncomment and run to re-render animation \n",
    "# # load in dataset\n",
    "# data = np.loadtxt(data_path_1,delimiter = ',')\n",
    "# x = data[:-1,:]\n",
    "# y = data[-1:,:] \n",
    "\n",
    "# # import booster\n",
    "# mylib = section_9_7_helpers.RegSetup(x,y)\n",
    "\n",
    "# # choose normalizer\n",
    "# mylib.choose_normalizer(name = 'standard')\n",
    "\n",
    "# # choose cost\n",
    "# mylib.choose_cost(cost_name = 'softmax',reg_name = 'L1')\n",
    "\n",
    "# # choose optimizer\n",
    "# mylib.choose_optimizer('gradient_descent',max_its=1000,alpha_choice=10**(-1))\n",
    "\n",
    "# # run regularization\n",
    "# lams = np.linspace(0,mylib.y.size*10**(-1),10)\n",
    "# mylib.tryout_lams(lams)\n",
    "\n",
    "# # plot round history\n",
    "# mylib.animate_lams(video_path_1,fps=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "eHh7iZfB0y-S",
    "outputId": "10915b11-b148-4998-a3ee-a3fcbb89eb0f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<video width=1000 controls><source src=\"data:video/mp4;base64,AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1wNDEAAAAIZnJlZQAAS6ttZGF0AAACrQYF//+p3EXpvebZSLeWLNgg2SPu73gyNjQgLSBjb3JlIDE1NSByMjkxNyAwYTg0ZDk4IC0gSC4yNjQvTVBFRy00IEFWQyBjb2RlYyAtIENvcHlsZWZ0IDIwMDMtMjAxOCAtIGh0dHA6Ly93d3cudmlkZW9sYW4ub3JnL3gyNjQuaHRtbCAtIG9wdGlvbnM6IGNhYmFjPTEgcmVmPTMgZGVibG9jaz0xOjA6MCBhbmFseXNlPTB4MzoweDExMyBtZT1oZXggc3VibWU9NyBwc3k9MSBwc3lfcmQ9MS4wMDowLjAwIG1peGVkX3JlZj0xIG1lX3JhbmdlPTE2IGNocm9tYV9tZT0xIHRyZWxsaXM9MSA4eDhkY3Q9MSBjcW09MCBkZWFkem9uZT0yMSwxMSBmYXN0X3Bza2lwPTEgY2hyb21hX3FwX29mZnNldD0tMiB0aHJlYWRzPTYgbG9va2FoZWFkX3RocmVhZHM9MSBzbGljZWRfdGhyZWFkcz0wIG5yPTAgZGVjaW1hdGU9MSBpbnRlcmxhY2VkPTAgYmx1cmF5X2NvbXBhdD0wIGNvbnN0cmFpbmVkX2ludHJhPTAgYmZyYW1lcz0zIGJfcHlyYW1pZD0yIGJfYWRhcHQ9MSBiX2JpYXM9MCBkaXJlY3Q9MSB3ZWlnaHRiPTEgb3Blbl9nb3A9MCB3ZWlnaHRwPTIga2V5aW50PTI1MCBrZXlpbnRfbWluPTIgc2NlbmVjdXQ9NDAgaW50cmFfcmVmcmVzaD0wIHJjX2xvb2thaGVhZD00MCByYz1jcmYgbWJ0cmVlPTEgY3JmPTIzLjAgcWNvbXA9MC42MCBxcG1pbj0wIHFwbWF4PTY5IHFwc3RlcD00IGlwX3JhdGlvPTEuNDAgYXE9MToxLjAwAIAAAB4LZYiEABL//vet34FNwEDta7pXONT/427Nis5TxsRSD0afgAAAAwAAAwAAAwAFolPg4K619GwJkBorpa84iMZmQATszelY7lJbsQPziiWpwhxretBChy3Ce3PQhZbMDqEK6LYHiGUz8V0h5S8Wr3LL9Y0yi6Cr7tWjIRJ9XqK5q4rEvKNkF31WX87dqtcS2BUdvvOHik6jnylH9awoL9gunO3eNTOMnodvTn59BAf35rpbMc23D680pNAxs7SBjvmCdLpm1Fu808mw0RvI8jM3oxKCNB52imScfnDSJGiPp2HRXO4ye76MKC4WZi39LAy+HayAK4D82NzBRk0cMdU3D6u9cVdX2A7mtJeOKYmoddvnp31Odac5o60f9t2M6dyArUdW/Dl+FMhA5JKeMblN4tK4nxj/k36QjGWELF+a8OVvooyCFR713A9Si0RaqRmEhyTozwso3PaZJXmlg/Gaw0rm30j7QVt6zzaPdusU9vbElSaQK5aS0GlWyJKZ/EgD14vVL6TRXOTw3GCd5bH5zM6+K9idmdXxsKgnYV7t0oIQUriLSe756sO3hQrjUNXmSvor1eZYkC2LVGFh+8ns+PSsY4J8sw0Mr01tWfE74Vm5PvOuHjRlIkmJoButkLWrKgcOHlqY2OvSF6bK/rzIWtl3EafM4L6Cof2MkWFVZmVSL3RaQu7zRQ4fc7dwDeUO5I3duEZHuNXJ6YBVUPdY8hwA1S0zM1lLLpiQBJOsrvP266bOU9ATvQi/lpWI1755A1u+yHZqUMkTNALJOUBhNqCrXWXN91fVK2d0imbL/T6L4YqRpni/E0KEF24kTlxQAz3sK5IAKXEaHsoqvgtN00x/5iyT/fq7CtcIUx7QT0PnFjcCG+kuhRpjFOIj/duOehQF6ZM6/HebXLvRndQTmVknM7+AhDABDPtI47wMoQHlRnDj9EVKS5GAkBkqdIkygbAKD5WQwcgcLAFN+LeTJIoiOgxIjMaqC2cVrXPHRK0ODkzMVgIpKuU3ElQX3TyVwSG7P3AibP36DCn4myLNBGKZrBirrsR7xq7E61ttMCdrVMPkT+P5RTjMirZRtuoV9gS629X78CjHKtDKdBmRpnhQZAuC6OZ/WyLTl4fzzMmpX18pFJDCWyN4RBumIEc/1WfVw33W+oOuNvGxgChZolORrRLN0CQlmo32cwbpyrFznXcJkyhSDjcbubis4yLYi8cFTsYflrDI4nypny+iSXo28mMgeRx8in16cU8GIYnmSKFiSg+UbGR0hJR7hVScPfrDuDdVLJiJxXwZLUTpnMvtZ00YTl5Gx5ZeVlk31xQ97+SPhL6obJO64L44HHz8DxwO0Pgqc+wR9/De0OI9zM2rmn5iONAgocgwcE4ARQIS6m14sMJT59H+LVM/GXYz9DIC5D6rnIJWROMyt9ccKQaB2tjrF5MLkT7/lI/IVSAX70IPzV5yjD9rcQsVajmuIlGbQF5JZAREbVolfiyWZj0R+En7xKD9FlPf6OUScD0penmsQ7vPFnnACI8rMkPiJkEHErKxjqq91h8fzwvNGVQhG4mYjH9ch2GRpvxmh+P6SvqXgj7IKEkPBV6omsRTVS70L5oadbL94nAS32IMHp4HBj570ePsFQ9BNXe2Cm5Gr7PSF5sA5Hlf4di4wqhzuHfIPYAfUspM0v+lHzFzViFzh3d/c9rCRBnT1kTbWpgByqMYu5PGAXhnvME53q0xAkMFjfW9RMVCefCiYdpqn6Bi6AUqDtXHGwMIEFGvyMn6z1LzftfJHlvBv7AAU5wdMY3ECJUbQIL/NFtg3/OiJEFkwHBE8zqFL2kmgslBRWp7GReproU7j+RSnLSXvMI0HekYKPybP0gveaJgAAApXg9D5k4JkjrTY8NLgtwBBdAvz8S5Ra0miulkk2O55UN3IgUwl9krN+jtciGAAAADAAOzhXY3oS+xQ+tnKzaw7sJdKDEWiH3zr3o8Lf9lGfEvII66pArBAnfBFjZsk2sx/E41BO3M8PJgNBybXdoj+aP74YGF71voJ1mlAQNwOGhcXlTibeNPpi7eP4fI/YOZVcXH3SjWcQcenojwILV+2aZaUZB13jzFp9fp8CLvI+vNYoDJIYhwlBtVGZGSFd3U0hYxTMotvPmPfbwp5BXwvEbGHZPtMyaOoMlhAW0vLe4zhg1dz9ZrtAAAAwOreivk8amAAcysQoew6L/WU1Gj+I5h4LhmIuq/ypsvlDpJ/7LwMrVsOWsxh7bb2lwU49373ijTjkuICXJTMpVbKuFXvC/6XMDkXuIcdveG2Bgv5B7EXir/7FKF3UmGQo3g7zN3NH9VR0NIQ9su7iFM32g4IB7pOS0ntWuWy+IX0kI9bVsl4ryA0U09fM0yrGDYrImYeK4132M9vruYDy7ZZJhz9M9tivU5kZ63Yh+t6B2W8iukVCxKiUR6z1J0evmn6gKjRnbiDZkR9HA0Yqt2Sgam/ghWfd8ZfN4r1Y8C79wjbT6gAQFoE+WULfS8Xt8g9nbprCp0/tzLV7M/8JfXlh3+AACNKxKB4+EJfft8YztlwCboEnlqpwSgp7tKG7urD89krpmD1GSkWFHwtg78hKhjZ/DenM1qS7SuxjbQTiOSQ0xL693Z2lpfpZLvSrZzMLXLLnjZrEwHbtFbCuPpqgkpWmgFFiQ2mnlh6/Bj4KAfjDfWl+SBOClugI7Jm97spRm/HAsu7NSO+zxQuzVV8f16UQDNDnE+bJ6ZmZcfbWaCbyX4QCyT5cfakMFjDjibLrSyKrxv7WUtRLrC2TdOvwaWDlgGBDGpz7mKbDdIBjkwlBWHBi5IqFIkyjMdo7YE5PFk3M/SmpNKKxS/NPAzElskClBOWFOWVkAAC+a7Z68hDfQ+8T9DNPgrKvJFTgWr0jWEVUKMWvWj+6jgYXSr8h/juvAfFZB//WmPCFh3pi6+NnM1IoXubhEMMBXXY2emlXmy20kTnizo8VHGBtYJDajA4khjf46kf674xNCQp1q7OTqrus5kmYauFIc5ligvEI1NBzT4MW8hyM8Ug0hpP9jGOT9FaXzuR+KAx3lPwNHEK3CaVvJMVWW33ysuY2H2Antk7VrQHTKtNGBjxtuVA4dE8uTz0z34dM5NnVOApcuTMVxuoQHMtLhTm4HPSjfDQur23YxTapKILt0ijoEglWkHyaQOx2B/xZAj2iwefu3pDfONUhLTerEA7hsTZ/8/kvorBmyqz8Mwz28hYhcTqU6GoC8TQSCzr1Prd3viDA/jjZsHEgLFI3s14v92ojg3CPCQ+J4FrZO2oXN5Mg6IpwvctxgXy1ckYfhGGDac4z2sOwunvt1cTwQ08u+QJTHwmvK4QPWN4M87ScL182ovss8GUdarOBa4nQoQB+0Exu6dgNq6VkgK/BsbxpYyXxfPz+06XFtFXXdsaxfW598CExiIp7AgRoKzmOfeQZjldqWW1ryV/wa9nZvG6zsXLJAD8Xnn2IiEToQ6T46s5WYysPHIPjKtrIOfeyLFyczHTxV57h+4NiS+PbxcZ6MhMaL5ukABXFkMf7/ri0DNmfrXXYHB8A6B4Fi0dZsyYjGMdwa+VPCAfgsv3GXxAANOzsRUU4ViYWCuU9ao92PxS97woGKua7Ug2p9WOB/qnvJ7QigEF5LL+xmQx7t0Gk1y+IW9s2CwVT//D/NjN9aLUL/7dd+VOFM89XY/0tskAhIZsFn1MtQ8/NMuAD0KWkmDJpQg/XlHJvLx03IFgXFTHhbaUhnXq8bD3lHrkgJb40kMo2LypOWebFp6vGkvxKMiw4uKX0P3O4MxAvvXjRgHGxffd6KDjgl4yTnwsDUG6uDvjRMAxNKM+Y/MAjJdCLLfJ2cSuUwR0L3R9pksmch52joHJqxSuNh/Nu076SM8mGNnBJVyxtmz9RhpXii4W/le0kksyCTnJhk7QEz2zHyAUcHykava2Py+gFkZDE3BOlvP6GpLcSIgths3wJ7uBg8SFgtsRx8499TGiW4IZskiX1QpGBp9DFZxAdgwWneHdmPP3rL1foQGXY7ZmBEQ8PTzNfZBXXC832LPlShq1TrE7N4/djIGmctog9mNW6Sulq1dmvgKo/IrCz8a0ze4ZCDIwKy2gPq0vn2UdV6pjVFtkXsNmsjM96Hdb+srzi8FywV56GuWkfFde81fRw6fEmEfMn9Ao8tQS2A09WgI4Qsw2hFT8/daP8el2hddr7qoNJz0OsTqNOSbrtgu8cgP7qSF/tQCywCbwtflfsK6HtaYf2pzhdpZCsj+4mgjizZT1wImadFQ6ZBDhp8FDedRHmEKSnqPtZmit+XHlZohL5fm8RP+aFO+fzg6mpF+ynprauV/Xz2EU8kSwga7xoQ46NRQT6DPbeaKgq26qgW4xnmU8gtVkLiqhSYqFVbZHzVc7+wpDNqojfcYbONyEQacryXy8fGpIbh+PTp3PiT1aQB6IW0CnVSDASEGYxG0f7sBg+xqodYxypWp9ZAAllUzhNIdCSjn90gbS8Z0c6rVI41ILXSbcFdV6vc3mMbrXTjJB8JSlFvYhotLLzQ0mEZ9kEGm3Yv4wL5mca7lGUmfVRSoTwDvhSBg/7a9yj18L0Kct7C16Z8FEzi4pMw3eglFtOsW575xPIDusRXz7G4PaFbwtbWm8gD+QGjDzGim9ldrlk1l0ziTexe8QA2DvTsykakO7T5NilViz6Q7QxyNeMQiAnWsykGAsIE4h/UBCoAAAAMBbizyPSUnsRD7iNd6IH8bmj70GBzP/Ya7unJSDkR//ruJmBYXNqSGXyE4Rh8291W1IR8DPh07bjxXhrKX8zpe1V95g30tMZ+Pz3tBKTfGs/4mInV/LezMMVT0zJFg34aRjMr8PNo/EIqeClsXNIpkUw4MWqMgtY6O/QUm1JWAadq2x6pVycyQUlq/LSmTgMWdVqFweGLb/zMPik4vC172BDhxFxA0JN1gb5W5hHph9TKdPBHZpOiDmoaTPcKab8rXUkTeYsP5e4q1Z7xQtyWVuMQM3nrkUEiHxvsz1noVyzHAXaJlPT83Z8O5GrwG7FFRZDLjqddbD6JggbbBjmem0czEE+TIWTAlL+Saz2DQ2y2Trmj/ZmO5UrH4Q5g5DSc1NSzGLbKf+NizX5CFtiRZ20c//RsnNJ2tIToozP5Cwtmgub0zi87/+1sHH0Q6IbABebExMRMxfF/k3OA63KJ02uTkFvfGhCdpRRsKKQLUx6cyR7MVzElZhL/F6QrYqCnDbK34yZXU6yzCxR6FkVoxaClR9YXiUCfQ54+vZ8Spq9owT9Npj1mtNrRi2H/XOWSIn7xdir8+m2yhCem3kPYgfT8VdKj9kxKuXuEL/nHQzg3IapR+X0ZBmxO/i+6w7FIoE3Eaz/+KY83xQtfj4jqWI+8xatqINtTeIy8t9K2Tg6BV4wa87YgnV7cX9OupLGvhfOMXub6/tu6caEqi/HSljqlDxpd7xArCHBjIVlWi2C7u04plM9bECsaUpWCWB/43seuqZwMHJVuy13yZQH7AZEQm1sbaF7EfjtmI6rN6PA0BY3wNOgPr9+ZfrzTVZjAsXz/77tLY1yuc2hKdDxI+GRh1x2EgGmnpK0taV6ERVkRxS0nwp1SFca2QN4xiaq50xW3z9fuV09LANg9lbkaEGO/QWqK9uUukd3LjBRxJQp2e6uMlPjwOFjuEP/uFyUvkfTCg5Xrm6okgtCzHnEMJK2LhYK/a2ta6ozWUezk5JJqf6Mjx/dpc2hvjAv6kfXdI78oaOUTbz3JhXA7GdfMotD4VMU1bRTqnI3KXoCrJORDR8L8Zt+UR3kk/A2X53KT0+tFMj65HN8oRcfNv1xzrAJcavwBIeuOwsYdKAs1so1lgMxsoAWUJDiuu1aiivUrUA1CHP/IRm97pcJb/xfQlgt3/kia8qt0Hov6inDV+PJKL25c+BYJaUgvc7EEmuwjsmZhxKrEvD0HfxyJi1a+dp4q1deEfKfXPZNe4na2LnZr8MIs2v2Wtlxy7Ma1rEu6pKRFQrhNF656oA8nn9LFXZvfOAEcXS9um7clKhVeqB6d9zFa7EPayYpCyLBNAP2U7UvHph9TKdOEBS4DD6TWiSdi0GdbGYkvzKO4EIJ29tJiyk1mRUs1ZhLqJ7q1np40WU+f/JPnwJp/02QcEUPXc7EEm0lgkK/bO2yS8QUJMgOOy4HVwhIqLaD4D1AuRHxjjPdWzs/0dRJuGkNQWMSVNFuCJ+GQ79OhtjEOCWGuBUnxKYfuv3bTrvdrw6ncNXbIQT0z3wvtSk2xF61Szl/na0TvBmgsFWAp4UXSeXV13LTWL6kIDG8cIimLXunkiCB+zLdEWGMq05b8nU0+I6LacLy7iCA6+2NTP+5UYICFirLGpH9CcjVShHjewI/xoWRXIWWqfY6fX1DSc1QQ0S0ffl8gQc4h2GYJk/HneL2CYiUeLXN8K8VDdtmSq+tGe/Lem8Qy88o4+Lx+kRY5dwJKFUw3dUlj19v+JcVWBw+qCiB4Mle0YBCi3PCibDfZ29SvQsarA/EZDcVo0o7LoNKEISGXgE1y2r/RD//9HuGGDuel2MOQRzAT7fHaYG2hJvR+208MXduxLbxOPrQXB70+r4NrMQHLeLBYmYI+2+q9Hl5qSI3pB0LYozR5tV6E+i3OY0SrkHyS7aIHVsL0LzDF20nNUF1MAdwNK1afR0fgs7+z7nv+DCKq3jjoUnVw1PaFpmBkqDCHswRWd6p5Nl4zTqlQxTc6UzUuN0gPDBDdY5WX/sFxFvuxrA0z/////gr0E0tjsimAl8m3+XO6SxX5rUvgUAccAtRi5ObXCe9TCsg9vaQdCDECbcOaqyByVml9FiE/f/9+W3HFoVKexXSMQAxZ5PhPbK4UPQw9m9HDmVC6H5AGUFxdadrBRIdj7yzUiqCd7wqA8RJ9WMId2Mo6Aibjgpqedv2MzmHvHT4BWt50HqcgLuplwumzHdw1K30BOdtf8pcXp1XLccEjIj8f29dLrWGqSB1vgCtIOvhcGPMw+zBtUd1zNtmnRfXtassXwgbFeUz4zAkrHHngxw8PRCTsMaWvBI46EGOZI6gsgqtVDWeMBGTAiBoyZjkMC1VN2iJTfO2+eXZOKfRVNGl0g4LGY6BPiQwG3xmpyTsMZlVI6hh8h+2jFOfQKpkGu/L1Wngmcnbu72r2OSvd8agXL8oHHWTx6gRkI+SmX3HTD+ifVW1pIfvqz4pNuDzAOrphc+YpY/MRG8BAqZnpRbTDpvfdKCIh5Q/niamMdKoIUsDro0kR+SV0FCZ6Dto2jbcPwoi4ujf1EqhiLid5pXepM39Tp0lQf8lgsInma7ahSWm0aMQ5+YteFXUnc/tCweimdWH/WIWWJWuOKfGFS7RUoZLwUhRLqD2AWmrqSzmJPwiGnl1mI4PZFJzRTxEqqpU1DduAmRipKGwa3BnkQuxXb6qeGg9cyzwiknXdaPON2x80fMbEZs2EfjSPj4Tt9wyrsvcIKIcwZxtTAXtZk2SoF6jKKeh20x6sz1rSzNvLSY+Hnq7GwUYtv917LW11CBxbpmak20QiBcA+En9TJ9+7E1jL0j5WuR1/CQKwTwULdET03PdzIysHbYM3H8Yf5sZZ/+yVvux1ro3DZ2dmjjecb+PUCzAA4KB4wIg+jVb2Ns/zIBYGYPc8j6xdWSEqFZSkD0XCCyR7xk6CGa+Gm20a9DuYYOS0iSCIjcEssVu6eGfpipZog63SzGqspGUfSlpzSGTC5Qjl3D02IPY+tEz1kZjm/YYiRTk3HJmgUTaAXq8hxxvYbaHKuHPOQa8z/B3l7ESo0MipWe5ZJrj+Dt+x4kz6m+DSOG1oPVG4XHfn1TCrv0sZYcxpwdvu/j6zautCZYIYu+x1kkDCQSMIPMTUZt4cLT2I/Lg/9XsnaN4TpY7sHNHJdkJ0zLL+Yks1BXcMKHzlcNjjMJuHyiOFHJVE4KYaX0WeoUQ8xLWxvIciW4MC3zvhtXjLErWCPJ+USAiRoH2CTrFIaGWj9Ku6/y+ZRd/bb2FBhLmYl1LnZcBtIo7/jBw5PHWZcx+1+0DSiTmvGzxAmNS+h3tG42vhqqGrLCCUTri110LE261X48FqMtkC93ZO2tGzr0b0pCRRdl6nyQCMCRIgCgtPTzciPk2U5Tfs6RBPcmSKEWsWFqE4RfIwGlacGU6gaAfSksHUqSAvOnsFWYzXXXdiEbQ+Y/Vt5O/MTL+HzBoZ+YHaGP5F1OwGjSPy2iQm2LMggkrZ/+qqUImdbB7C/jmUl+UDznDGWCGafiDwrRbqWKeooRDnR4XVb1gxCrnC835KyjF50Tt4srb0Y28jKoO9Oe5hUmmBc6Qat8TmRduFfE98O0SJWxeriGaZKIntr9ce3AdAFxNh0GT7w8dRGuOOYPZW4SS6Lc4v+ZExvAVsCQbcaJedosnzY7lpFuoSxpDkLEqO8WdRzf0tLIJo7/xMYLfGjW2J+34YX9gvOnt07kVoy8Pzu5+cJS9TGExLgJl0f3HZd/7gdx5l///6flENXnmFHvHo9iPfD8wAR+IoRwLJtabj8fMusGuwGp5xG1EMx50yYuOhJINNfKTMSukFgjy+JNR/PHpXLTkdcjTLtHBDDaKWcs1/k/okMKKZzdih9nLQWmfx5+ST5BUjz852iNeL+F2KheVWb2ExV8sJj5qHRu+VV9kTGltvZn1VHHl8q5Ex1g3Mx30f4DkVnOAQ8D5XdDWDQyJ7bZKcUy7OpK6v/Azv/smIjQHJUTsm5jsHizAcDbhYy/PSYO2CiQkKyZaED7ZC6gUSY0g6FIllOXByA+YT/4w9VqqJp7ou+HONw6ddD9J5Ca3IwVkupLYmpp2ZYho1y7qJxPmeXABmj0Qu+77ALZbMce/VM0QVMp2fLx+X53OCGlv1mY9T6MUSfMbvKQbuq0p5WMwtCNKKgajLnu14cmf2vUNSL8Ae1bPmjWQS/MroM9kZIYHYURbrN59VWB5hYddZJ+KFLQ0+yUXetNYq2DnNZSxZwAYGcEHlmivFyaYJRW2kgXe+rtHzdwppUebZ2s92MvvXfMA4UdAXiSLT1x4jlhEzQytqyjRNdG+545IDoNYMYRE5hCdD+TlCQ56eSEFbbs9vNlGftns0YPbjzCG+wrxcYsu5bnrOE3IvdEIJYCK0BlYQBadWkPfmWtNadpwwwI1e7/RvVqCxn+HAEdlTKhyS08u9z4HWKrWcYRy7OKGWSTAtiK/U1eeybwtkwM8r4BWsQZNSkNbt2+bMMrJ4SrQ0yT6PaFbjsqLqgGGCxjh7zhj86fFKx3HexyvEJBCAkIQcOc8RjZyfkpyzx2/DQh8wI0L4hvYriZ75Ihf47vfDK9A+Qgsj+2TJhPJctWhtatgajfCjE5Uh34orA1D7/22CKdu0P2uKjsFG9dDx0xJmgLBw0xi2pQzXlsBuqzPq3Q1/b0S8bl0SRAnWfjjqZU+EwQCO1bvfwII+5kWcM2Bfla2VWH8pFRTJ4XttbY/p/gcrzP+1aLwtcME0CTwapJ/rzvHEZI7lbbJa9wPnsj53g3GQfYq3C1ZPs6Btn5kdaEtQsN4ZyHfBKq+Xz621AnGUlYvaTe9padtgDPg6xnibYaa8PdTzm+mMDPvqdlV6sHBmBqkSXJJwza+ICYR3fd5XW/Q0H/EFibqV8glWKhax8VzFjGAMR0V5gkt7UZJcfDvPIC4HAbpNKo2Zf4LTs+LHXUBUBjtLQfs5FokbLS8mzknpjXToqdQ07WjAC1+wY3/hjPuptf50CL2TpIAzFCEmJmkTrQpdD+NCACy7mP+g0ctF0bz4qrSfJyAYhgNPyyFE/RwCLh3tetpjjNoSSQ2uy4pWZtjemTa6AkF41I4D3WtJXRsIu/4RXrhkDnARznyaR/LO+k1I4zH033FiUTpKutN96WspnFYIxO+TlCQ56XPtVHhcxibyq/7++NTtxABWbfXRc6YUIzRZ8ZY+CzPxbi8IxxMCmifA3r80bo1RyYziFpcL+mLryYEz6mD1pQcAIHfzB4KTExrKtOegg4KyDISkx73bNlWLB9ex/h5p/W71xf77+Z5Pzaz1qPK655jhQJ6m0tKgsdObwjTXJd0jZ/PWLpcSgr3Jc/z/w12pmdtgdbBBNRPoyf/OKdLRYuSDatfOyW0SPIqgovySGzK+qDWRCEOtrxSQ5zO5dGxu1rOZ+TR6t/YJX+mQ0p+4vSZ8xqFkPifErcOsCH9178dY62XnCes0sXI6Pcep9qiI5QQboF+W8wACwcpPodEEAAAZAQZohbEEv/rUqgBAeav4rRSFAZP9BaFSqcUqZpeP8/3/FUUAWsXNipd1zwb8TxoeaiQw276R7nE2khs3rpGntGoCxWZbTfFCQjq3ZRSwolZ7YdH4CFk5/hqHSitGTlAaA+I0efYVeVVCVnmjeJ1/ThBhbf5OgJZLSE7hMHxzMIewvQ/6WldnJ9kTxymJTuzfeIDau2oT+ei6VODTckchUGwYM3uWoqiMYuIpbmYgNSBIAdeZpnaaetftlunCzq/F6k6fMHJ6ixGfRx/KXVZTLP9OAO2GKg0XkWFcWQXUnE7pWwc7hRMpxD5v2fDvcZN78RqyakvTDP7g8qKTUf85K2y5EdjrvO8+fAULd1u7fdGM9WrQZwOylnYK35aQ4Da7GJV1wEnQdZzfqQGYa92gTAZcZ4lLqvbT3DzfBjX4LpOD9/Z48fufNebZtocDZnOQWne4tPiel2OLEJMffLI/eCMfdGWGv0vzkkvR4eBud+UCMgAei+DS9ix9Ry9ejYmQgpRGEGSfwuATl1/+qgUJFbBvEigYA1ChWGHHFyNGG9yq+jIUfb5dnNUaVaWsialwc8v0bw8JOlSC2GKJRlAkmXGWPcyhAa7+fkB7ngR4e68bgHuNtbxMAidX8C2X+jPWf1U2enLUH+ymzaGCS/RmUXAi77SjCrc5bxcrBWlwLBjtvTs3NJMvdA/+AGPVIf0r/LZZKZ5CXoj6CBxac1pRVl9qWcphJk/UY60GbsIeN/6HODZ14I5pp3CxNV+yRdtQLCCWYCMPShKPuajIk075Nyh9wGbHzmyBJYEG69ELfvLci80VmeTHQhzgYMuW/zFf03Jus6gQDWoYDkc/7auRVH2NAkYhdyzWd8daUIT90rdv9XV+TnfhZLnhqeSTP4I6jRo7f62kLC86e8Y/UPabiR6NF2B4dupubaUQfxz6DgkTU5Js8SefSZdf7hnvL7K3zFN6uVQImx5ipnA7lbSMwBOnnpp+COsQDiKWIlfIO0NKEemIv//GpH7DBCM82pMaepbxYyLxdeFrXzyWbUiSf8mcx9NxzVJRFamSynkyrie5lWkJN9BbcD1VGj1uv/VLnne4Bp9BKm6U/8LKLW+1+XaWMeP1laMahjf3+7kIbfwmQiQxSxEJJRAYcKFlKJtThE/O9a0Jlforj3FAh5uJKZGjp2L3uNjnCn5jXer9At0q/8HbOsVn3Zf2GNpPscgQ8m4zNRIv6HI+TNbVQ1eWtpDY9b/sr5MFzYrDM/eQlZqrypnq4RvHT9z4doCKjyQ+MJn1m7ZqXVy2qfGTZgQ5f87rij1QcCrtzbyF6aiHXzgdeW8svZCPxTV29EFyqbW1YAHOVj+ahky1uaW9G3VT7wo4IcQ7pKQjv55VJSFPIk6mWKMW/H+xvuB/s9o4HYThOTQ5I4LMJOon55+Jj2b/dc4xkjFpSkBJn3Qj0CEmVXQcnQhN3oagq4mdX6B55JuTf4VWmvPtvOjjl+Hi/lzxYnSRqlOZ8AAmrT8XRRMkc1aU+xrQi5cOAe/sjofKJ2M0Omj0VGs1jv+az8k2HOnz3Mzcxdn234BONjGikKkiyxbPkG2VDW4o4aOFCh8rjGwb9h9ueE3fK1jAywU/smDZawK+/IwDnKsqOoP6jECHB/TWXfjWRjX9P6aBr5ikBkUCrpOQuJyE9Wi/QiUdL+X1od9cSBLttzRWASdNjfuc+B8REdC/S+EgXwO7XMKp0VxllCqwSsdt8MMu1abrMg8dvOrH+l+A7UK8xeqUpC2F9kDTfvcqhRknjSMPWlhZkBaDYl2geK3gLtyTorsy/YuTWOItVF1GPcGmeHIguPViz74LBjefXOmMWxSSzPjX284RBnLruFYgTb+by8Xn27HMPuLZenDS+Exhwi4FBNsUHyWcyGo2xDng/JFIbxR1xGsN9cNszzkfmIs4gnogFHT6MY7neUBD2TDaL4o2zWeKyYos35dBmWn5+Ejbx5+K49Rjr96VN1oYtmIWsWUKozrVFY7Q6UbxqqBC4SHfHYYViAhAB7/r4C5Yzt8L7O/8jwSP+A78ObVJ6SIgwhd7tjH3bzCgru5cEASi+2nHlH4pEwzZ5GVY2qVl8lIylM+cxx3y5gnSQOQi10fsaAPvWKmTScAAAA4hBmkI8IZMphBL//rUqgBAGnn9QBtwnIEt74/GsdLllFdHlFzBKAWzKkJWNSEke95gMbGfNqOv1icQLbPdc0TtYdP1qnk0+eCR+jZl8tTO5RfTDEOyoRVBLKy1j4eZCQkXKugMeDeYN2w/LTxTfy5dQ7aUhPh61Wpniy3MHvv2MEnu9rppKN6V28CfdDyU20UjgUnPXHb6HyLwzA8ypgIJfCQMu+WbdiDEmOTuU31h+pfJ6CQaW/pU1qndWFRAkYs/sE5urhqKzjnG0N52wF0w9oVgyScBNpjx2B3BdmGnz7q8tHx5d3+ue7Q0yanI5FFv2IY2/rpeTt2eSojxJoELq55rHECf0MNai9n8ks99DUT7Y5waDg95TdSR3JR5mvgCdIachE0MWMQIMQAiUIRPF4JQdYJKYB4E96m03uWmXfyc4fdW3y0E2pQPIvDkNIVpY+ayweZ/fNaM8iFw2AQPFQ0LBDV4Edmk0isnINaP4VOGMIEPAVqcWZsTlYunfw84rX+DikFnF0hqPo+f4O5zDgr6e4igBVVToQpds29tbNrBMpsvlHSvgK+0BEN+VOdAxb+1o1BJc0bGmqV3EXEfMSRzT5QQYvYXS9JZk5CAY6lFxLB2+jTW3vVR8fUwk0gzjI0xOJYkFXpzAwFiTBCggrDoc9VzgkPo4y+sJYpkiHltUzlw/plwe2qczYGuvpg1D6HiO4Ht2uegQlDfupyzp8+SByQJE7T2+qYSpI5I65qUcgOZRrOMvlxsZlLDLHOPG9Qo/UdJ/FYEVtXmQmIrzCbW2EKFRGzhc+S5QAADBoqMvE/ttUIHi4/IKeEVa+F9xXX2U8NaEP+mDzRdppoU56v6apCaGPwmGltqQfK4FWeKjzx4if/V1db2cs1y0Aktd8GKBOm2v4EKPAm5skHQxISKto/Qa3coGuBBxmcoAbhr1griiR744mY+JlcBSJyqrg58b2d3MFqx9P4jdjBm6ExHI4FVpJDW4oJuS4NEo4BjTeArFjYpI5WC1QCmdoHmktoeteATS9ghJbwPIsUlPMTKEcidmjRUPYdtQRb5Y41IZ0IQ0i3iN5qrfTTD5dDIGbMB4P3P5LrFOvZeQuLnp2s5pAmH6aJ+xs6BdiyBoGuwxx0D+IfGkdNVAN5c+J8ZaJVbkojHMdjMSVSFCanvW80EeVVkTBSN0roF8jONxMAu7Ukv2iuTBAAADV0GaY0nhDyZTAgn//rUqgA9KSJ92AAtl3FjBvzOX0tgrleO+cI028sfymWKHtpoanejz8WJudKX/FEgFTR3bjjaYmDVrVy53VIud+7kUiSfRYKUl8wa+xA4X8Ad1FNq9MtMbgiJ8FMlxbJ2DFOX3ubEQbTg7Gbt1Xhp4TcCYfrHKdtUozEKj9vBiV+qtzRL7ArUYgY9kQSqTqdEVPNsW5U/WEy9n6T1lsz2qe5nSU+bX/plALZCY18+kbjS2j4YlLLph4zcqFxT/6cByOnSqPfPt5TyiQ04kbdBd38mOMbaZyNuAk0GyfreWzsZ0TCiDwfCInvr3Otz18SwSKQWyMnHXeq2/gBzmLylrWjZZG8duY3AV80xHALuestEWZsY0nxqgQV37jok4+kb1BSa8gXbz1+c4jnm3N5d+y0yMDhWhLiaXaODp0DCI6zBxXSjDS6HLbD3nQozkLYcZhEZMSCXyvTPRyUzKa1Rn6b4JjyhTNa0ronA4O2ZKuAc3GDyYsF7znWk7PvXuIjVIh4wiqoaQz2EuJYGHLYA57WQRB+VzW03rDImm7OBiVtLlbjoEwwMPu1KO/zLOlb8CXaFRbnk4/J++hCE9zVI0cd+UqFNo0zBWhdqN/s/OjwOEpufYJ81eaKyayDH76pCIfC6H7eHcR3x9V3nBRkn+K5puarso2EwplmxnYWXeRtD64M/WTBh+3N7MrgLCKJ+5L75fjSzbefmO52qndlgnOvp6MURNsheVGI/aONeXZ2SNEzVtGh6+DtwSJSvOlA/UmM6v/nJihNrnd0I1qgBf/7nR8iVq4qfe2XQxw7GZ+Mxua+OlTmj9A5DzUNRvOvCqnswJABAH1oFeWGAe5ME+cTf2zEITkJj5OCXQZVpgiehheBptQhAbjVxjWstWN8WrukUxi+ud8Uk4B3b8ZbYpN2FmTcLnsq1QCad1ETZ2Av01Xwix9l2gGCF6aJzP/oIv8e9l7zwt/VtjMkLgfk2riupQ1uF7wQREzQeIJi99le04pgzGmm9U9qN0Ej0Tq7BbBFj7AUJBvSHwrtOHPOl9I3ZvTbWSu4Yq4ZRcGbma5cOJCnj+F2AmUODor9awSKT0QIyaZOTrW/gAzq7H1R6PN0ySzj7bMT42hh6+ygAABJNBmoZJ4Q8mUwIJf/61KoAQBqH4JIPuhzwDAAmdCqx/nSWSw4SV9IrMVgmLiBkEIgtrbrW0EF1PRV+FTelZwTlxneTKT2x/ggVVSV871d3OqU1UA26URDuojU75DguyWH7+b/11T7qvxFu8pzvyYYBjI7517cc6l42wfpRCbA+c4cLjBFxkymiQzI04k9xPnh82+TwsASPt/e0U4n8oMpIwupxBkcFDNIPs/E3SwPa67WdtJaJvBLrwhBo8bs8/zyQAAUW0WicBTk1wvnfISwWA47QX21uZO7+kxyWbZjEzl/2vAv5kRomtYCOYgRw2psRoRBfFugU2fcQA96rECZjLEnO1PNG0Q1rwECHiH4SB8dPkOc87azIiKmsK6Mycxx0MPoiHH3t2OhE0GvAWB/1PyqZuZvrTOYbXSal/ztNEHOYKuMKr3gmFEewChJEWs7fSbkG+NRizd04CkFYxje3iD1v+x+aUHcatqNpiCj/Q+OiOIDoQJGFkVkHY+/7godUEb8hZWESGCpeURV1jP5Pbu0JDqraK7Sd9jU6xlvIqZqb7iKgTLHyi88EYsBhVDXV/ZyLnqpdcvj7+r6wPWS2V+ldzmeTrag3ySz9ltjLncwbYvIfDMu5+zB+Bc9WeDvzoJAkiHaC3YGlPr7LXm61R7V/V7dtTvjkgy1khCj0Dxnyw0WIfJsZ11Z9DRY56/5bjAVjM34C+2qchJUOgc6nXxyBP73JEp0Ah+ZLfm7nCRreUt2umMVw94PldGPciypnez4U/5tgSwwa5PuZXm88KVYEMFWAOQghSTmPqatE3OPo7QhOOADHATZ1lxKAY4Wm9KYnkB/44QVl8Oi+O+XEE0TtXMFShI0leEAjXIxn58NEFW9TUWTVvved/hL7wQrp7M22/jEwA9F+npprj7IawXxYvhPWcDew0tOUk2JmvIaQ6ZGjva7Tav9/cbBtF4t7u/XiKvFr8y2HLhpArVrG5R5ym5KYx2eOLzB6/fieDloW9yJ6Mbl7nepHSzYwjbPerDe9Z/0UGahG4IO2+I0e4xwkYyigKN2RBraJ/u9rjHVxCEBDnH3XOH6X3RzyNt6Hp1z6hJBtVO+9bwqxtzPldQwsxLVHOPPrTs8ANi4pzmE0dUSI0Q5G1GTjzqXVrHYNwiFa3OkzXhn5MBuiua5cU5PmNvYPgvIktGYgLoAXraH8UxR92KkpMpQ+edMUzIiRgAAADAAADAASBLIMk2wu2p+6AAAElJrKKtSMSkddPeqlBKvh6TfAnV/llfrqCL/7k9g07gn5D/daj7nLv5boM0sJYo7fWSRfLMsu/5P3WKaM6MVEU+Ua9M9gnH6eKwlyewuro/qwMiVYgi8vWgtV3AAF2qZ6Z3QcJx58y5bYU0vL11zMESDvdAOonP0+a2l51oY6YLzewbcjhZjE2i/vupfV9dFT3dWpgZU45bEbIv9opLyfoEQdNyE237ex9sLz/yz7lYMovaWC7y3nHFNq6czgqHRuzTn9Ps+Jqz1mJiqUWqpf+XLprlONx8wg8I1F/Oh3ISOTUaEvINhQv2BCWAAYFAAACaEGepEURPBP/AAVuM5FOvt0b1ElACmPgBEKVaj/r/u4TME95W/fj0sOCyXx3BDuX6K4ax35Gz2E1WnTbzmua8BHG6uDfdq4MoSd4R9bV3KyQNUbJ5wgtVLqDopJvta1msBb94JOWVh7cewHAuqhNlzEqWfK838V8NFjSB+thlxoM76KUfdcGm6+ZEKWZqO68YoWAlGtl67+i70AYVY9ABGl+EqOK1jmnJcnrrC/JdLyzUPWht8FbAHreme2DaIVD2n1QVwVou6MrXsoeRJvOMoUjvdzw1n7lbCaeuq+99ikVHUAvDOxTthsPnCLnAKvQxQ1cQarhXhhFXGjAgpzxi9ThYQmOy0C8zoOBvdG0BhMETknkB46YVJbe2ndRZZsiR87fCBrYvnTQQf8rEBna2ZgFNQXDexq1vsfA0So8zULFTYSWfaH4VN4vLkWFdRVw8vfWkDT2cS3qAUnSbcDmHZF+L4RQhuSCt1rREPnMGLWkwCWtwmhP6xRFIjvQgFXum59wTKvg2N2V7WYYO7vf6qtFqNQsjmK3EijAmifqHCOxDI4nS1NomJWCEVF3ITSmRVd+W6I4AXsLc0W3H7IpZu1VPK92nYDOLSZ9gpQJyEGS3PH1iCeCLPN05PLCr/dhz/QGqFwh/YkLt1VoCETk4yPpOCn/CbbB+5zfrnz9nwG64DFNnUAAAAMB+IX5h1BrZ2Wp08xmE4K/ijrUFFmbdhjZLqIdrmOnb9YzHv3st74D6ahPyNb0zZBNzqLR3yAHV2VUAAIX9mFtohGtUywDH28nPMQ2IoSzv/Pcnsq6Q7aAAAz+RRYADjkAAARAAZ7FakEvAAfwNeiAmm6C7TL/tnkDI6gAQhukQjSfgWfWZXQDsgS75Iyj6/4c2/lf0jNHcq+fTGSL8GzZvtWC5Nb+TmZ2iH8ttmuybaj12M/6EbJDKbK/hiSE3qsNm1IAPx3xzwfQzmzGDW8s9iBUrtc78JORXXGQfi1pLCHjwsX6bT6elFg7KWJdjGBJPwj2DgKgzm1iMO0g9fmrMo+mYVcOAF0ZGC7xtVc38wDnRvqBQVIhqUBTEFwLmuQ+mVnefBWigpE9SkaW5UyAiumST/yNPmUepqXtjDYEqeRN+tnjqIhj3SVSg2GTLuQ5mW/zDFozlk51+MRxq2EPfGB52DTvLZoDnJRCc9HDepeZQNh2MiWd7fs7dNRE84KWktrTpVKHXbC/9loZGDafLIbivjguozQN7mmrXtkHDe/WL44RGy4ahXHHCKfyqTGN/kgd5pOq+HFikMtTKn9ifbrFNm0Qjvb1r1rMUhqNHzIJAsWxPChCSrZ+hPXVea4rF2mfwhf4FQExQ6o+3gYCC2fZk1vuC5bnCsaHPMSVf1ileYqqmf8GWykyETt1L+QfXZ68+5dlkAZeyXqhjlZ3wao/MvZO4RRiNMtAavD1BM/ZN5319dkv27p+ZEFRH4+P4Vj+XP0XIi1ZNSQMIDDc60iIFA40jvgomYMcRTpbCPv6bCW2bnOAI0SVBgp7M/2AsK09yZAAw+wmCjEayAxV93slbvKsquH2CYTwcMwVLPpeAyaaPISIphzk9Vn+uZ3oLFJkfwe65i8yFnFnYgzcqINbqG0fMSclbyYlpME/jg0TGiOOv1va9YeaMaCCFpEBqhftvYuMPzKFZ8p3ix9tOajutO8G1gcCF96amJgudoA+OYG0IpUrO/MQkSt0cEnCmwuSwG7Vf0gj4Bree8Pga76wme+5CuhFBXMrL6U6wJcAMTS2XqCl+x2AW2WMuIydAy8tZu4ZU5WlMjsTlAI2wB/dzBOP/swkH0WIsq/Usb8zip6Lhlr/t42X2X4tmyI7ZkJBM4EnR4h279jcvULQ4XyiYxFcsYnM34HI1n3SVuZhxFbY3TUsMlAw2jj3K2M8Mghf/YFAAY7TULs83oD8U4qyk1u83YioaImjjeLoiVE4HTwwbvLW83ospPYnEqiFx8I3rJ5vJJtamg7DzmCH0hEuI/bqR8axFPDl0t/h6nCFkbBPA5olJpAeDlfc1LNu1QUV1hgK02ErD7OqktKPucMXtUyxPBm0ay99gBhsSBlwuWC8G80Ubh1/UfOz0WgAAAMAAAV48ljjWyw6XSev/9TIIhR4otXJvsUF1vP9RMi0Y5kqtrsWr25Gmy6ZG05Xb7OTqaksApEHZ2lj83Gq7IcZgLE/9/KdUd+0tFE5wyKeppur/7igT6Nuj6dYVwPzHNL2H9+l/z4wkMYdaLWoErgPsc2sBvaJakLeHYCbPpKYC2kAAAW5QZrHSahBaJlMCCf//rUqgBAFHK1FeAzCpv4AXLDu0bqGzy7x+1yY8xiCWz6nJoYpeN3Y6SZ8nbQbUXHgUXuiQsUnGT7r9wkwKEnWx1aISiIDg8YcTORRSNaT5iRtutegz3rVdE3GYhUBYhBLaWkWn+ZCPH6ck1ADWPpXUOB4i7r8W3wZDGuEyyvCrU0bO05VAH05MYNUMAQWZX5gBZMCrmyZKKEjECFYvHkgzO0QZsOFdQ7e3nOL3lSNYl9GQ2QIVoKbIbgumiqToppzZfyGaT2AlfcLlMGxf/QJqLqeNMYnM2NrqeByFOxuwDlHj84xmA4Rkyg8ng+SYbObIAW071An8qjgb7lNvhItg6RvEoB5R843nZVKmpqWJMzc2IySgoRyZL0BSOwIf5mxJpz+8HPCyCayZQMnAqQjgbcAgu6WxXr3d0d1Uv4uPQJUuShW4RXjDm7XlILTfialQGhlPLiFIdJUT0ZsjML7sDM4Ay9nE7G31BqVEsRHDzC2XkCbEersFtjwGCvaXPiKdmVjMYDA+jCp4MJretnBGtHeG/Gceb+iA9fflXR3LzIcVK+7cN5pixLfQ1tvJmS9eBVMJMvDSzcrD3zfQe0UUx8rfFsWAkzj9pLli0/KAKudM+EAmIKeQ/i2j425WU4uTS+1JhztKe3BQZvDJxWxNh+uBZdh7eOlSIw038AaO46Qrz+CaCLEzANwhgoyev/EEikQ2Jrsh7QV7S9ROKe5ycGeYxN1qgGZN7RAT+C7VCIzly0NHMseHWHdnyWQTP0uCeLXICBN8HFj3yLsCh5/8f+b9UpbVBql9KCQtXEX78me/5No2YVtKD4GXHHP9geGWq5yXj+Bo3321TSMQoshcgs3R0Yno+/ERyp/HmbeIxeuxOO/kA20K/gCN0USazXQgKovf61m+Yf6UkSM7+mJrbuJBdByR66ANYOtN/gKdAxQUBGDU5hfXsQ8MzhLW7rK7oHnIE6oShG+2+k6f5cZAW3WWMJt09sTcsC6x8IcK9PR5pGPyWbceJczyNr1awmrkILeuuRQRkXRqDAzlRwbWogZKKDLn52zPUw3w2eYjS2Un/M2Twx5FbvyDEfRtuHFS12fIiHCmx9FHIILRee6VV8zKAxijvA/vziYCxZXAQ+yP5Mr9/I5mGoJACAKL5bXBZ12v06hg3e7qLspp8boSsD4ifREgVcwVympp8d8JEMC0dH0V1uZw/p1goB91atYC54TvHk0uwZKNJqvZMkajWmnidDqvNUBAEh8GoXUIfT9nQTanUIDuEXkCTO/aklVyDLFX6nbkl/PvlzjFAtN7DnE1U387uLuBy1n+BH50gz4Pjknfk1lVWtCX7G29NhQkDHKv7V6LFPQWiwq4KqQuA6nNd1o3CsX+ww2buf02QwuJiBKW1OPdlflwpBpQL8M6teMtV5QHSgmSDknJgI8fANYUCMHmU4wyP3KncS0w4RbLx8lHmQMncUi8QTi+mU6gWgQUXvDBGjFjsyEbdwA2v6UAeudlMQaS+c4SuNNo1jrhtvtHleSxLI7iiiOy7J+TVIAQGmWOeurZtr7Y0hDuNpI1cBw3LLkFxHIK+be+wdkCuf0nY+ADsM4gfKL5Hc+W8lX9o0eu2iKt9lxePuwBrGKm5XO2bisoJQN7nYoOzfq+n2FamJW5Iq90lCpDgM5+xZKxNulHAgv/Nth3MOyjNtYVd5oxSk8nySQ/O9rSOTFwz5J/DXBUsQJTEe5V0F9yiDcrUevy3gt93HcK1QEw8zP66PylE/fMM9eYKO4iBOybdRqx4ZNTGXxU24oFO7dJvgMv89LdlXPm8ydWOJ3aADRC2GMjopCkeIi0PMEF8YSqptrNcA6tx98ilrWCYjvjXe344wRSzkqyoOhZ77NQidLgcytAsp5P2PsZMK+fyrjznr2VBHzv0alP+K65U73BVaLG+iKh8wrsknF7QAACR9BmulJ4QpSZTBREsEv/rUqgBAGnkghM+g3kIAIdQqdkkd8xTMPLF4odvUBPPkqBKb2wFK1IydyEBDDix6N31yNztaXgpofzYqY6jnRSc1DsO5RSekpyZJAhdtpYoHZahSGhxCsoxNznUHtt/T/njE9ZYpLALPGE0wg7Qh2gAOxVVRg0LUXpV3M8qViJUBjNVUrYqbrTDvT6IB5hl2bPpNDWjVvSJHbNfdkDrgEvAxGADlOjslLJXxb3Bt0KxMBsp039+zEKMNXp2cQ4g11KZfOXd8U9oGt1uHh4YpT5dna6zXnDJhaXnS7ykCvk5wwHULdsz0BEaXaZ0WIOSkoQQd/CPCaWdmZJciif+hU8dO/Er8/5Bw28N8qK1md9+EcvSYuXADInokqhaZbQEPsfDKdsX9ivBOiXEh02MfiKzvILHBZ07AlGdA8+xn3xNXEGTh9/iW4hZdRGRnDXt9whDxyWIaXfJPNmrzVzNkHiAZtm2vjnXWt7U0rzBfzygucAzlpGomWHWJrRLvqIpvRHnharMcQ+xua8oCKYH2rZavfU3OVr7tjyf3JQg5n9uhG2I4lxSnZBItz46nzfeF+PcvcCpq7qx+3+j2vEo22SaI+Y1Laey66LrVAIWN0el88oDBAMGTT07P3CzwdSni7he71PLfv4/7m4ejV1hMjVGd/mRXDz+6XCxepcqJG9R8uAON+5a+EX8HUoFU1ULX5swO4Qt4aGx2koxCCr7YdfK2r3uT714gOOCReCRbeQt/YlgBkRQR8Eyg3CToca9QjibwqpZslKULHQjukTQqbFCUfBBitAyGCvu8Zd5yzjYM9I8+HRGw6gJ7d3VLcDnnHcR7VnB/NHr5WOTFDpEOVKY+SLOQzSU98EPuO82Lv21Tbgf6G9WOQXZbbZCqQwvzdGww6tnjfLbktyh1+t992h9S3f45+YC7EDNN0EGM7ifubCiFNwMN8CVgz3XuemWbU6Zdr0gLJbBbcG1gNNcLD1tgCMdf8T/jZdnexy6v/fKjkadByAS3dx4+i82jO2mSGNN0uPo2rnyeiUZF+U8agp+dFz6no16Ntrh5lNhvTtrQObwOKqHXmq5fcXf8BLVsDRzFU8q18os7dJF21sHohgFpzHBeJBlyN/tpwFkPObbwusigybqgb05WzIT0y6StGXjyE9xVyCEGguCx+aaOV8P+joYOW+LTLIvaN+oMelNKAGHgCj4V27MwX+clXPU8Njotro27YKHdf1qaZcWd6O4CI14qRSWzvdi3z8MW09d/xdyY7L+kSr+q5QVrbUxUqypxZuAbGHhrQyqslaLuL/o01aR2yG47Wop9kEpc2f4IBVcdO15U1XvQ/jNUWYEOv7ULRnyTpSuPn0DgebysbtAeFkdbGdHhzoJB1Juj/1bjIyxgJ5P3NzxgqubTTTO1Ya46zTl/2on8BP/ne6HEwlM6/gGZeKk7wS0/v755b1GqXedRN6FbXgW4mst73fO/rYYEzk/71uQjBa7jHzIBeLCJ1q1EcfuOV2bTrJwXJjUmdD2uxd9kPN8n265A3HNckb/mi6lUuODzZJVXC7QNEq/flofE55jTkVtDLRWWiakcSTj+XPq8olkPDVj/xcV8iCDHy0C67a24XeZL0B/QgpITwNrjtP04jmsHLLSvY9DILwy5UlZfCEbldPS3dF/BevimF3Qt2qyP6MjMzvZexBrDRnyi28WRKEYTABiTtKsPBGTBS2pVZvYQg4pHmH7A003dhX/yqQcIrL6a4+S5vZmad79H6H8jVsj14dFCwrVDQlu+bduXpkOlkQjVp9qFyZs9zK6yP9R6TXlCIX7MA+JhCNyGmRQK/8AHd01bpMSPZKuGq5D49daWA9+EsEzVnXf6uSBiU+FNMl6X/I8cTA8GWJwly/3LMy5W45T73BOANNCt7oqltLI7snRB3iV/AmEyQoD/tXS+eTxCL/KMA/0IFBG5c1iSS/2392NjPDHA5F/4mMrdKAxMuiFKz0HuU+y3a5dHsivX+fgwIHzLdwJfTdiq+kxP+uBjgSshTdCAxCPOosffsvl1q5Veo3ttpOk8yNyJxvDGZ5gJ103EeH+8GrHPlq9dvR96AWmZSVkflNF4o4gFw1sDQAAADAAADADKwQiff6vYtmICeKy2yBWXkypKK61RyvujYDYTjMRW7VMgCuHRY5RNBNMr7lmWjKyx5sq1un7/h3ObKrcWdxs4ZxBUaDepbacYXLhyB7hEZt3/S54DXCJW8C1UABcAn4G/dpYO4o+4PVcMwusBlr84GpvQSJ5eLm+U3ga+jZ1K9VdFU3zsiKFIb/fvJKPZ6ueXIneDW4fQuFZ4Mw4yCY0VjQa1yZZqLIim16RN5Yjs9nlV4AVlAP2rNPTpQZpnXWi82WheQ900n/i1MpE67WpoYqgwVugz7kYek6S/us0X1/lajDe9MFlLHyTmKtIuKlMBeOB8n7vMgNgSO7l0k0rpuvyeGuIcsa13r1R2n7Ruz2x79GaMIEAdsdv++r8PZcWf70qDuVVzAPNDNNuIvJ89N/8TCBUlaFw+90yuiNzmZNsFfLlWpT008B22gNeOQV9nN0/p4KoVaV0VI26oOaY/6kIvAn7gic7PPC34W0F8BpbdynBRAWdnu4AfNX3UMq2HB4wW+5HPrrbD1ay3XrOxLve4tB40hD28M8I41H4wIN2pBSlxznE2h/TCYNfkUz9ho0mDXvPgJj5JE2Ak2i+wEMSSjhtTi870s0e75KsdO5yRhaVSzwFrRlLqz5GF2yDrY+2hEV8mQ0NwnPHVAji0J/lfMdjbyfT7yac0/iD317apytfpyADgmC7UfLGLy0bXqf9PqWZzixpOpYFPqKuHcJmxpHN+OCu/uwCFDTcgu/iH027AWogb8pSAyWRZWbj/9/z/S3qTH2MJf0p9xkbJkmPTuoml1P7/f9WZueFBf1nUAdjLr6kZXCP8/7iXhfhz4HzSbQqpUt8f8kUo8u8U66cb3SjOsn3hnNcyrm/0yPfHyiNsQW4DRGVfLNIbu0iBdUgpMhe/YcgG+kSJTJJcISB8tEfRMa5AKSeHN0QJGtuwKIhojRfEyGe5bRv+fmcGwecZ2zgeHttFZ5/tsnGZAAAADjQGfCGpBLwBeZQsH72e6jwnbG3NAKgAQiNlthr4U/tke0EhDYyBEnyTfrfInuUMCDdftY6D9D4hEVstbckvi+JW5El3JOcI20rbkWV6XvycFFnCqh8xQMvrgL8qBj6E734uqX53xNE2TA7AtEZEX+/EmyfutiPP2OIa1Ih5TDxB81MqV1sYSGma6YC0vRGUwhL+NiTivDG/FI4FEuNCbaO6wo9fPEHJhJ1+qTWGHswv1SfATGUa/uXmxDeecSLCQHcL9K7H8uDqt9mgpfI8D8daPCLQmcmG5DrYLkTmDv77jPG3Z0oTxuLXERn9KthC1CSzQNB+kom4WZAXZIk5mk6tkbAUJVjEsxTdypGr+Rm5ifOl8HM5wuBdAdYf8MqVNNE+9HNmJp33OBxK3SubrU5FzQHgJUAwbCG78bojgF6Hx5lYfA2T0C+CwcqvYUg016GuoFkSRGV6h4mVQ81gk9edGsYhHT0jF/HPAjBHxD8QrNUXlghK71H/y1mV5yUGWu2gLfyq5WHdb9Yd9B88JsfMNVkR3j0df/E1cuJ6oc4jWBUrgVTix4RwoPof4EhxEhSP9k+TE9IwaEn00W2EMBQ5PTa4bjjIkVQLJKPsUlQZ/2K4wWkIkz1rgQc5ph1GR+9drWAKDV3U43rAeD7mRoAXhCp2rlZ4j4lSETEqs4aGUmZ1IO6FZVvMVFAFKe7SsrCCtDh8V8iyZnTCktjKi1/6IPWrrDgAAAwAAAwB18eKTj9T2svFx2y2tnGW59r9LgY3nY/yHaQMjRfigHbK0f9AE2x0azBFEn7IxCIzdTfLrDTbGZGf6tGGIXIGwskx4+kzlGSJEOm3kw6GZ8NZE2dQEeF+EI+eJyS+Hhk1uMtdAciFyaScXkLTUdIh+gaHUdV1QjZH5tEp0V2kW0Zz/KW5Ax4uDnbi2nVEERqUpjlVQdKzwEXx41MRO02zFVHwSaqUITYDscJ1s5LP45lrpANeMdVDuRt8eogi39cw1/VjWxo0sjb+7nUTOD7Mz//+xQrXywxlXtO9a+xz7PqdOTBGRZkkDbUlITnFWZ+2crQgzRCUDN9wKtWMgYcyYFVTKiKschV0QUjjarmsqX81pTyrrIaSURG+Z/sxDZbIIz7LADe9KpL1ynv+qNnRasFmWNNQYdViymDQVmtX1Sk6slHyjGtWZZ0sUG9I2m/aYvQAAAwAf0P+tjs3cE/g25AAAA2xtb292AAAAbG12aGQAAAAAAAAAAAAAAAAAAAPoAAATiAABAAABAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAClnRyYWsAAABcdGtoZAAAAAMAAAAAAAAAAAAAAAEAAAAAAAATiAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAEAAAAADhAAAASwAAAAAACRlZHRzAAAAHGVsc3QAAAAAAAAAAQAAE4gAAEAAAAEAAAAAAg5tZGlhAAAAIG1kaGQAAAAAAAAAAAAAAAAAAEAAAAFAAFXEAAAAAAAtaGRscgAAAAAAAAAAdmlkZQAAAAAAAAAAAAAAAFZpZGVvSGFuZGxlcgAAAAG5bWluZgAAABR2bWhkAAAAAQAAAAAAAAAAAAAAJGRpbmYAAAAcZHJlZgAAAAAAAAABAAAADHVybCAAAAABAAABeXN0YmwAAACZc3RzZAAAAAAAAAABAAAAiWF2YzEAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAADhAEsAEgAAABIAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAY//8AAAAzYXZjQwFkABb/4QAaZ2QAFqzZQOQn57hAAAADAEAAAAMBA8WLZYABAAZo6+PLIsAAAAAYc3R0cwAAAAAAAAABAAAACgAAIAAAAAAUc3RzcwAAAAAAAAABAAAAAQAAAEBjdHRzAAAAAAAAAAYAAAAEAABAAAAAAAEAAIAAAAAAAgAAIAAAAAABAABAAAAAAAEAAGAAAAAAAQAAIAAAAAAcc3RzYwAAAAAAAAABAAAAAQAAAAoAAAABAAAAPHN0c3oAAAAAAAAAAAAAAAoAACDAAAAGRAAAA4wAAANbAAAElwAAAmwAAAREAAAFvQAACSMAAAORAAAAFHN0Y28AAAAAAAAAAQAAADAAAABidWR0YQAAAFptZXRhAAAAAAAAACFoZGxyAAAAAAAAAABtZGlyYXBwbAAAAAAAAAAAAAAAAC1pbHN0AAAAJal0b28AAAAdZGF0YQAAAAEAAAAATGF2ZjU4LjI5LjEwMA==\"></video>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_video(video_path_1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KMYkAOyL0y-T"
   },
   "source": [
    "##  Comparing regularization and boosting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Mu499hOU0y-T"
   },
   "source": [
    "While boosting is an efficient greedy scheme, the *regularization* idea detailed above can be computationally intensive to perform since for each value of $\\lambda$ tried a full run of local optimization must be completed.  On the other hand, while boosting is a 'bottom-up' approach that identifies individual features one-at-a-time, regularization takes a more 'top-down' approach and identifies important features *all at once*.  In principle this allows regularization to uncover groups of important features that may be correlated in such an interconnected way with the output that they will be missed by boosting."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "include_colab_link": true,
   "provenance": []
  },
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.15"
  },
  "toc": {
   "colors": {
    "hover_highlight": "#DAA520",
    "navigate_num": "#000000",
    "navigate_text": "#333333",
    "running_highlight": "#FF0000",
    "selected_highlight": "#FFD700",
    "sidebar_border": "#EEEEEE",
    "wrapper_background": "#FFFFFF"
   },
   "moveMenuLeft": true,
   "nav_menu": {
    "height": "216px",
    "width": "252px"
   },
   "navigate_menu": true,
   "number_sections": true,
   "sideBar": true,
   "threshold": 4,
   "toc_cell": false,
   "toc_section_display": "block",
   "toc_window_display": false,
   "widenNotebook": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
